One of the primary functions of an operating system is to manage the memory resources of the computer system efficiently. Each computer system has a high-speed, randomly accessible primary memory, also known as main memory, physical memory, core, or simply as memory. Its access time is on the order of nanoseconds. A program may directly reference information (code or data) that is resident in main memory. Such memory is relatively expensive and therefore limited in size. The computer system uses a number of secondary storage devices such as disks to store information that does not fit into main memory. Access to such devices is much slower than to main memory and requires explicit action on the part of the operating system. The memory management subsystem in the operating system's kernel is responsible for distributing information between main memory and secondary storage. It interacts closely with a hardware component called the memory management unit (MMU), which is responsible for getting code and data to and from main memory.
Memory management techniques use virtual memory to run application programs (i.e., processes) that typically exceed in size the physical memory available to them. This situation can arise, for example, where more than one program is resident in memory or where a single program exceeds the amount of physical memory in the computer system. With virtual memory the program is given the illusion that it has a large main memory at its disposal, although the actual physical memory may be smaller. This requires the notion of an address space as distinct from physical memory locations. The program generates references to code and data in its address space (also referred to as "virtual address space," "process address space," or "process virtual address space")and these references must be translated to physical locations in main memory. The hardware and software cooperate to bring the information into main memory when it is needed for processing by the program and to perform the virtual-to-physical address translation for each access.
Most memory management techniques today use demand paging for virtual memory. In a demand-paged scheme, both physical memory and process virtual address space are divided into fixed-sized pages, and pages of the address space are brought into and out of memory as required. A page of physical memory is often called a page frame (or a physical page). Several processes may be active at any one time, and physical memory, often being smaller than the virtual address space of all resident processes, may contain just some of the pages of each process. Process addresses are virtual and are divided by the hardware into a page number and an offset in the page. The hardware in conjunction with the operating system translates the virtual page number in the process address space to a physical page frame number and accesses the appropriate location in main memory. If the required page is not in memory, the process generates a page fault, indicating to the operating system that the page must be brought from secondary storage into memory.
Described above is the typical situation, where the virtual address space of even a single process may exceed the size of the physical memory. But in some situations, the physical memory can exceed the virtual address space. Programs such as database applications with large data caches can have a data cache space that is larger than the virtual address space supported by the processor running the program. For example, the PentiumPro.RTM. microprocessor from Intel Corp. supports 36 bits of physical addressing (64 gigabytes) but offers only 32 bits of virtual address space (4 gigabytes) to maintain compatibility with previous processors supporting the same instruction set. This combination of a 32-bit virtual address space and a physical address space of 36 bits means that no single process using conventional memory management techniques can simultaneously address all possible physical memory supported by the processor. Yet it is the existence of these very large physical memories that allows database programs with large data caches to exist, keeping a sizeable amount of information immediately accessible in physical memory rather than stored with delayed access on disk.
An objective of the invention, therefore, is to provide a method for an application program to rapidly access physical memory that is far larger than the program's virtual address space. More specifically, an objective of the invention is to make a multi-gigabyte data cache entirely accessible in main memory to a database program, even though the program's conventional address space is smaller than the required memory. By providing such access, the time necessary for the program to access the data is reduced by several orders of magnitude. Although the current use of the invention is in support of multi-gigabyte caches, the invention can be used wherever a data space exists that is larger than the process virtual address space.